Systems and methods for virtual machine resource optimization using tree traversal techniques representing alternate configurations

ABSTRACT

Systems described herein may allow for the intelligent configuration of containers onto virtualized resources. Different configurations may be generated based on the simulation of alternate placements of containers onto nodes, where the placement of a particular container onto a particular node may serve as a root for several branches which may themselves simulate the placement of additional containers on the node (in addition to the container(s) indicated in the root). Once a set of configurations are generated, a particular configuration may be selected according to determined selection parameters and/or intelligent selection techniques.

CROSS-REFERENCE TO RELATED APPLICATION

This Application is a Continuation of U.S. patent application Ser. No.17/142,854, filed on Jan. 6, 2021, titled “SYSTEMS AND METHODS FORVIRTUAL MACHINE RESOURCE OPTIMIZATION USING TREE TRAVERSAL TECHNIQUESREPRESENTING ALTERNATE CONFIGURATIONS,” which is a Continuation of U.S.patent application Ser. No. 16/787,989 (now U.S. Pat. No. 10,915,372),filed on Feb. 11, 2020, titled “SYSTEMS AND METHODS FOR VIRTUAL MACHINERESOURCE OPTIMIZATION USING TREE TRAVERSAL TECHNIQUES REPRESENTINGALTERNATE CONFIGURATIONS,” the contents of which are herein incorporatedby reference in their entirety.

BACKGROUND

Containers, which may include operating systems, applications, etc., maybe installed on “bare metal” machines and/or virtual machines (“VMs”),in a virtual cloud platform, for example, which may include discretesets of hardware resources of one or more hardware machines. Aparticular set of hardware resources (whether referring to a “baremetal” machine or to a virtual machine) may be referred to as a “node.”Nodes in virtualized environments may be provisioned with varying setsof hardware (e.g., varying amounts of processor resources, memoryresources, storage resources, etc.). Further, different containers mayuse, and/or may require, varying amounts of hardware resources.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A-1H illustrate example configuration generation, in accordancewith embodiments described herein;

FIGS. 2A-2D illustrate the example generation of multipleconfigurations, in accordance with embodiments described herein;

FIG. 3 illustrates an example selection of a particular configuration,in accordance with embodiments described herein;

FIG. 4 illustrates an example of selecting and configuring a particularconfiguration, based on input configuration parameters, out of a set ofcandidate configurations (e.g., as generated and/or scored in accordancewith some embodiments);

FIG. 5 illustrates an example environment in which one or moreembodiments, described herein, may be implemented;

FIG. 6 illustrates an example process for generating one or moreconfigurations, in accordance with some embodiments;

FIG. 7 illustrates an example process for selecting and configuring aconfiguration, in accordance with some embodiments; and

FIG. 8 illustrates example functional components of one or more devices,in accordance with one or more embodiments described herein.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The following detailed description refers to the accompanying drawings.The same reference numbers in different drawings may identify the sameor similar elements.

Embodiments described herein provide for the use of machine learningtechniques to determine an optimal configuration of nodes and/orcontainers installed on the nodes, based on input parameters. The inputparameters may specify, for instance, configuration criteria for adesired configuration of a set of containers and/or a set of nodes. Forexample, the configuration criteria may specify a set of containers toinstall on a set of nodes, parameters of the containers (e.g., resourcesconsumed or required by the containers, or other parameters), and/orparameters of the nodes on which to install the set of containers (e.g.,resources associated with the nodes, such as total or availableprocessor resources, memory resources, storage resources, etc.).

As referred to above, a “node” may be a deployment infrastructure forcontainers, and may refer to a VM (e.g., a discrete set of resourcesassociated with one or more hardware machines) and/or a “bare metal”machine (e.g., a set of resources associated with one particularhardware machine). A “container” may be a deployment which utilizes thehardware resources of the infrastructure in which it is configured. Forexample, a container deployed on a node may utilize the resources of thenode, including underlying “virtualized” hardware (e.g., hardwareprovisioned for the node) when applicable.

As described herein, machine learning and/or other suitable techniquesmay be used to create VM configurations (also referred to herein simplyas “configurations”). A configuration may be an arrangement ofcontainers placed in one or more nodes. “Placing” a container mayinclude installing, instantiating, etc. the container on a particularnode. Further in accordance with some embodiments, each configurationmay receive a configuration score associated with the utilization of oneor more resources in that configuration and/or other factors. Asdiscussed herein, a configuration score may indicate, for example, theutilization or non-utilization (e.g., remaining) of one or moreresources of one or more nodes, the number of nodes used for a givenconfiguration, and/or other factors. In some embodiments, the generatedconfiguration score may be based on performance metrics associated withthe configuration (e.g., processing speed, network throughput and/orlatency, etc. of one or more of the nodes and/or containers included inthe configurations). The generated configurations may be stored in arepository for future use by a configuration optimization system(“COS”), in accordance with some embodiments.

For instance, as described herein, COS 101 may receive informationregarding available nodes, receive configuration criteria specifying oneor more containers to deploy and/or parameters associated with nodes onwhich to deploy the containers, select a particular configuration basedon these factors, and deploy the specified containers across a set ofnodes in a manner that is based on the selected configuration. Theparticular configuration may also be selected based on one or moreconfiguration scores associated with the particular configuration. COS101 may place the one or more containers (e.g., as specified in theconfiguration criteria) on a selected set of nodes. As discussed herein,COS 101 may be, or may include, a device or system that is able toprovision resources of nodes, and/or is able to install, configure,instantiate, etc. containers onto nodes. In order to perform thesefunctions, COS 101 may use an application programming interface (“API”),such as the Kubernetes API.

As described herein, COS 101 may generate, and/or may simulate thegeneration of, multiple configurations. As shown in FIGS. 1A-1H, forexample, the generation of an example configuration, as shown in thesefigures, may be represented and/or conceptualized in the form of thetraversal and/or generation of a tree. As described herein, COS 101 mayincrementally place containers, associated with a given configuration,in one or more nodes until all containers are placed, until theresources of all available nodes are exhausted or full, or untilcontainers can otherwise no longer be placed in available nodes (e.g.,due to affinity or anti-affinity rules).

COS 101 may, for instance, generate one or more “tiers” of the tree,where each tier includes a set of branches. While discussed in greaterdetail below, generally speaking, different branches may representalternate arrangements of one or more containers placed in a node. Eachtier may be generated based on a root, where the root includes aparticular set of containers placed in the node. By way of example, twodifferent branches of a given tier may thus include the particular setof containers associated with the root for that tier, and one of the twobranches may additionally include a first container placed in the node,while the other one of the two branches may include a second containerplaced in the node (and not the first container). As described herein,COS 101 may select a particular branch in a tier to serve as a root fora next tier.

For example, a branch may be selected to serve as a root for additionalbranches based on an associated branch score and/or other criteria.Generally speaking, these branch scores may reflect a measure ofefficiency, performance, or other suitable metric associated with eachbranch. The traversal and selection may iterate until a tier is reachedwhere no new branches may be created (e.g., when all containers in theconfiguration have been placed on a node, and/or when all availablenodes associated with the configuration are full and/or cannot otherwiseaccommodate more containers). As described further below, a particularconfiguration, containing a finalized set of branches, may be identified(e.g., based on an associated score) as an optimal configuration for agiven set of configuration criteria, and the optimal configuration maybe provided (e.g., installed, instantiated, etc.) once selected. Asdescribed herein, COS 101 may be, or may include, a device or systemthat is able to simulate resources utilized by containers and nodes.

As shown in FIG. 1A, for example, COS 101 may generate a first tier ofthe tree, containing an initial set of branches, based on a particularset of containers and nodes. Other configurations (e.g., not depictedhere) may be include different sets of containers and/or different setsof nodes on which to place the containers. As shown in FIG. 1A,container list 103 may provide an identification, order, and resourcescores associated with each of a set of containers. As further shown inFIG. 1A, node list 105 may indicate an identification, order, and noderesource scores associated with each node of a set of available nodes.As described herein, each node may be associated with a particular setof resources. Nodes may be described or referred to as having a“resource score,” which may be a score that is derived from, and/orotherwise reflects, an amount of resources associated with each node(e.g., a resource capacity of the node).

A resource score for a node may be an “overall” resource score that isbased on all of the resources associated with a given node, such asprocessor resources, memory resources, storage resources, etc. Asanother example, a particular resource score for a given node mayreflect only a particular resource, or only a subset of the resourcesassociated with the node (e.g., processing resources and/or memoryresources, but not storage resources, etc.). Resource scores, asdiscussed herein, may have values that are different from the resourcesindicated by the resource scores. For example, a node with 32 GB ofmemory may have a resource score of 10, while a node with 64 GB ofmemory may have a resource score of 12. In the example of FIGS. 1A-1H,all of the candidate nodes have the same resources (and the sameresulting overall resource score). In practice, and as elaborated onfurther below, similar concepts may apply for nodes with varying levelsof available resources.

Each container may also have resource scores, indicating resources usedby, required by, and/or otherwise associated with the container. Forinstance, the resource score for container C1 (as indicated in containerlist 103) may be 20, the resource score for C2 may be 32, and so on. Aswith the resource scores for nodes, the resource scores for containersmay be derived from or based on some or all of the resources that areconsumed, required, utilized, etc. by the containers.

FIG. 1A illustrates an example initial rollout of branches, inaccordance with some embodiments. An initial rollout may be the firstiteration of branch generation in which containers are placed in a node.As described herein, each branch may represent alternate placements fromwhich a particular branch may be selected to generate additionalbranches. For example, as shown, when generating the first iteration ofbranches, COS 101 may place (at 1), or simulate the placement of, C1 inN2, utilizing 20 of the available 200 resources from N1. As furthershown, COS 101 may place (at 2) C2 in N2, utilizing 32 of the available200 resources from N2, and COS 101 may place (at 3) C3 in N2, utilizing28 of the available 200 resources in N3. COS 101 may select one branch,of the first iteration of branches, to generate additional iterations ofbranches. While shown as generating three branches, in practice, COS 101may generate more or fewer branches in each iteration (e.g., one branch,two branches, twenty branches, etc.). While shown as placing onecontainer in each selected branch, in some embodiments, COS 101 mayplace more than one container in each branch and generate a branch scorefor each generated branch based on each placed container. In otherwords, COS 101 may generate an iteration by placing multiple containersin each selected branch for each generated branch. In some embodiments,COS 101 may select a single node to generate a first iteration (e.g.,place one or more different containers in the same node).

As further shown in FIG. 1A, after generating a branch, COS 101 maygenerate one or more branch scores. Generally speaking, the branch scoremay be based on an amount of resources utilized by containers installedon one or more nodes associated with the branch. In some embodiments,the branch score may be based on a value derived from an amount of anode's resources consumed by one or more containers placed in the node,such as a resource score associated with the one or more containers,and/or a resource score of the node reflecting an amount of resourcesavailable after the one or more containers are placed in the node. Insome embodiments, the score may be based on one or more other factors.For example, as discussed in detail below, these other factors may beused in selecting alternate branches in subsequent iterations of theconfiguration generation process exemplified in FIGS. 1A-1H.

For example, as shown in FIG. 1A, COS 101 may generate a resource score(e.g., a score indicating the amount of resources remaining in a branch)and/or a branch score (e.g., a score aggregating one or more criteriaassociated with the branch) for one or more branches. In someembodiments, a branch where available resources are fully utilized mayreceive a more optimal (e.g., higher) branch score than a branch whichleaves resources of one or more nodes unused. In some embodiments, abranch which has remaining resources exceeding a threshold may beassociated with a higher branch score. The threshold may indicate, forexample, a preference for a node to leave a resource remaining above thethreshold so that it may be utilized in the future. In other words, forexample, leaving a relatively small amount of available resources (ascompared to the amount of resources typically used by a container) mayreflect a relatively small likelihood that another container would beable to utilize these resources in the future, while leaving an amountof resources that are generally close to the amount of resources thatare used or required by containers may reflect a relatively largerlikelihood that another container would be able to utilize theseresources in the future.

In some embodiments, the threshold may be based on machine learningand/or other techniques used to identify a suitable threshold (e.g.,relatively common or frequent amount of resources used by containers).For example, machine learning and/or other techniques may be used todetermine that existing or previously generated configurations tend toinclude containers with a resource requirement or utilization of 3 ormore, the threshold may be set at 3 (e.g., branches with nodes whichhave a remaining resource score below 3 may score lower than brancheswith nodes exceeding a resource score of 3), and/or the threshold mayreflect that remaining resource scores having a multiple of 3 arepreferred to resource scores having multiples other than 3. While shownas a numerical value, a branch score may be represented by other scales,for example, a classification rating (e.g., grade A, grade B, etc.)and/or other ratings.

Branch generation may occur, for example, based on received criteria.Criteria may indicate, for example, an order for placement of nodes, anorder for placement of containers, a prioritization of one or moreresources, a time limit, an iteration limit, a number of branches togenerate for each tier, and/or other criteria. COS 101 may performiterations of branch generation until reaching one or more limitingcriteria (e.g., a time limit, an iteration limit, etc.). In someembodiments, COS 101 may be configured to generate iterations ofbranches until COS 101 determines an optimal configuration can begenerated. An optimal configuration may be determined, for example, whena score associated with one or more branches exceeds a score threshold.In some embodiments, branch generation may cease when the difference inscores between different branches falls below a threshold. In otherwords, if generating additional branches would not yield more optimalconfigurations, then COS 101 may discontinue generating additionalbranches.

FIG. 1B illustrates an example iteration of selecting a branch andplacing containers in the branch to create new branches. As shown, COS101 may select (at 4) the branch with the most suitable (e.g., highest)branch score (as indicated by the dashed ellipse encompassing the middlebranch in Tier 1, which has a branch score of 32). Once selected, COS101 may start an iteration of branches in a second tier, in which COS101 simulates the alternate placement of different containers in thesame node (i.e., N2). As shown, COS 101 may place (at 5) C4 in a firstbranch of Tier 2, may place (at 6) C5 in a second branch of Tier 2, andmay place (at 7) C6 in a third branch of Tier 2. As similarly describedabove, each branch may receive a branch score based on the resourceutilization in the branch. As shown in FIG. 1B, the first branch mayreceive a branch score of 43, the second branch may receive a branchscore of 54, and the third branch may receive a branch score of 45.

FIG. 1C illustrates a further iteration of selecting a branch andplacing containers in the branch to generate a new tier. As shown inFIG. 1C, COS 101 may select the branch in Tier 2 with the most suitablebranch score (as indicated by the dashed ellipse encompassing the middlebranch in Tier 2, which has a branch score of 54). COS 101 may placecontainers in the selected branch (i.e., the branch with C2 and C5) tocreate Tier 3. In some embodiments, as shown in FIG. 1C, COS 101 mayplace containers that were used in previous tiers but different branchesthan the selected branch (e.g., COS 101 may place C4 and C6 in thisbranch, which were previously included in respective un-selectedbranches of Tier 2).

In some embodiments, a container may not be placed due to anti-affinityrestrictions. Anti-affinity restrictions may be attached to a pluralityof containers that cannot be placed together. For example, in someembodiments, a container may not be able to operate in a node where thecontainer has an anti-affinity relationship with a second container.

As illustrated in FIG. 1D for example, C13 may have an anti-affinityrestriction with C7, which may be indicated in container list 107. Forexample, container list 107 may indicate a grouping between containerswhich are in an anti-affinity relationship (as shown by the letter “A”in the anti-affinity column adjacent to C7 and C13). Thus, as shown inTier 4, COS 101 may not place C13 in a branch which contains C7 (asindicated by the “X” on the double-arrow between C7 and C13). COS 101may place other containers (i.e., C9, C10, C11) in lieu of C13.Anti-affinity restrictions may maintain a reciprocal arrangement. Inother words, anti-affinity restrictions may restrict placement when anyof the containers with anti-affinity restrictions are placed. Forexample, if C13 was placed instead of C7, COS 101 would not be able toplace C7 in a branch with C13.

In some embodiments, a container may be placed with another containerwhich shares an affinity relationship. An affinity relationship mayexist, for example, between a plurality of containers which must beplaced in the same node. In some embodiments, each container in theaffinity relationship may be placed when one container in the affinityrelationship is placed. For example, as shown in FIG. 1D, assume that C8has an affinity relationship with C9 (as indicated in container list 107by the number “1” adjacent to C8 and C9 in the affinity column). WhenCOS 101 places C8, COS 101 may further place C9 in all branches whichcontains C8. As described above in regard to anti-affinityrelationships, an affinity relationship may be reciprocal (e.g., if C9was placed instead of C8, COS 101 may place C8).

While shown as indicated by either a letter or number, affinity and/oranti-affinity relationship groupings may be indicated by other means.For example, relationships may be marked by a symbol, letter, number,and/or other means of demarcation. In some embodiments, affinity andanti-affinity groupings may be indicated by the same demarcation (e.g.,both indicated by numbers, etc.). In some embodiments, COS 101 maygenerate an initial rollout (i.e., first tier iteration) based onaffinity and/or anti-affinity relationships. For example, in someembodiments (not shown), COS 101 may place each container in an affinityrelationship in the first tier of a node (i.e., in Tier 1, COS 101 mayplace C8 and C9). As a further example (not shown), COS 101 mayinitially place each container in an anti-affinity relationship in adifferent node (e.g., COS 101 may place C13 and C7 in different startingnodes).

After multiple iterations are performed, a resulting branch may includea node which cannot accommodate additional containers. A node may notaccommodate additional containers, for example, when a node is full(e.g., may not have resources available), may not have enough resourcesto place any remaining containers (e.g., all non-placed containers havea resource requirement exceeded the available space on the node), mayhave a different resource available than containers utilize (e.g., anode may have available resources of a different type than the containerrequirement), may be restricted by anti-affinity restrictions (e.g., oneor more containers placed in the node prohibits the placement of one ormore remaining containers in that node, all remaining containers to beplaced cannot be placed based on anti-affinity relationships, etc.), noother containers remain to be placed, and/or other situations. Forexample, as shown in FIG. 1F, COS 101 may generate a branch (at Tier 6)which cannot receive additional containers. In this example, nocontainer from container list 107 may fit in N2 (after the placement ofC1, C2, C4, C5, C7, and C11), because N2 does not have the resourcesrequired for any of the remaining containers. That is, in this branch,only 7 resources remain on N2, and each unplaced container requires morethan 7 resources.

In some embodiments, COS 101 may select a second node to generate aseries of branches (e.g., when a first node is full, and/or cannototherwise accommodate any further containers). For example, FIG. 1Gillustrates selecting a different node to generate branches. COS 101 mayselect a different node to place containers when, as discussed above, afirst node is not able to continue to place containers, and/or for otherreasons. A second node may be selected based on a branch score, a nodeorder (e.g., as shown in node list 105), and/or other criteria. Forexample, FIG. 1G illustrates an example of selecting N3 when N2 isunable to receive additional containers (e.g., as described above, N2 asshown in Tier 6 does not have enough remaining resources to place anyunplaced container). COS 101 may select N3 (as shown by the dashedellipse) based on the node order (i.e., as shown in node list 107, N2,and then N3) to generate an iteration of branches. As shown, COS 101 maygenerate an additional iteration of branches by placing C12 (for abranch score of 233), by placing C13 (for a branch score of 231), and byplacing C10 (for a branch score of 228) into three separate branchesutilizing N3.

Based on the selection of N3, COS 101 may continue to generate branchesand/or tiers based on the placement of C12 in N3. A completeconfiguration may include the highest scoring (e.g., branch score)branch utilizing each node from node list 105 and/or may include aplacement for all containers from container list 103. For example, asshown in FIG. 1H, several iterations of placement may result inconfiguration 109. As shown in configuration 109, N1 may not beutilized. Configuration 109 is one possible solution for the placementof containers in nodes, and other suitable configurations may exist. COS101 may generate multiple configurations based on generation parameters.Generation parameters may include, for example, an indication toprioritize placement of resource(s) (e.g., prioritize Resource 1, thenResource 2, and then Resource 3, etc.), a number of iterations togenerate, a number of configurations to generate, a time limit forgeneration, and/or other criteria.

In some embodiments, a COS 101 may generate one or more configurationscores for a configuration based on one or more criteria, such asresource utilization (e.g., amount of resources used and/or left unusedby a given configuration), node utilization (e.g., the number of nodesutilized by the configuration, etc.), and/or other criteria. Forexample, a configuration where available resources of a set of nodes arefully utilized may receive a more optimal (e.g., higher) configurationscore than a configuration which leaves some resources of one or morenodes unused. In some embodiments, COS 101 may generate a configurationscore for a configuration based on the branch scores associated witheach branch constituting the configuration. A configuration and relatedconfiguration and/or branch scores may be stored in a repository.

In some embodiments, branches may be selected based on varying criteria,and/or different branches may be expanded/selected in order to generatedifferent configurations. As shown in FIG. 2A, for example, COS 101 mayselect a first branch (shown in the figure as root 200) to generate newbranches. As discussed above, this branch may indicate a particularcontainer, or set of containers, placed on a particular node or set ofnodes. Assume, for example, that COS 101 is configured to generateadditional branches until a particular limiting criterion (e.g., timelimit, iteration limit, etc.) is identified, and further, that COS 101has not yet identified a limiting criterion.

As shown, COS 101 may generate three branches (e.g., at a next tier):branch 201 with a branch score of 10, branch 202 with a branch score of20, and branch 203 with a branch score of 30. As discussed above, thesethree branches may indicate the alternate arrangement or placement ofcontainers into one or more nodes, based on the arrangement or placementof containers already indicated by root 200.

In accordance with some embodiments, COS 101 may select a new branch(i.e., branch 203, in this example) to serve as a root to generateadditional branches. As similarly discussed above, branch 203 may beselected based on the branch score (i.e., branch score of 30) associatedwith branch 203. As shown, once selected, branch 203 may be used as aroot to generate three additional branches: branch 204 with a branchscore of 35, branch 205 with a branch score of 32, and branch 206 with abranch score of 33. COS 101 may select branch 204 to serve as the rootto generate additional branches. For example, this selection may bebased on identifying that branch 204 is associated with the highestscore out of the branches for this tier.

As discussed above, in some embodiments, COS 101 may traverse throughthe generated tree based on the highest scoring branch in each tier. Forexample, as shown in FIG. 2A, COS 101 may traverse through branch 203(i.e., with a branch score of 30) to select branch 204 (as indicated inthe figure by the dashed ellipse) with a branch score of 35 to generatenew branches. As shown, COS 101 may generate branch 207 with a branchscore of 46, branch 208 with a branch score of 44, and branch 209 with abranch score of 50.

In some embodiments, a branch score may be modified across multipleiterations. For example, a branch score may be modified to balancebetween “exploitation” and “exploration” in selecting branches forgeneration. Exploitation may generate additional high scoring branchesbased on previously high-scoring branches (e.g., based on branch score).In other words, COS 101 may “exploit” a high-scoring branch bygenerating additional high-scoring branches based on that branch. Incontrast, “exploration” may, for example, utilize relatively lowerscoring branches. In other words, exploration may encourage theselection of relatively lower-scoring branches to generate additionalbranches by, for example, modifying the branch score associated with abranch (e.g., a branch in a tier where another branch was selected) inorder to increase the likelihood that the branch will be selected in asubsequent iteration.

In some embodiments, a branch score for a particular branch may bemodified based on the number of times the particular branch is“traversed,” modified based on the number of total iterations of branchgeneration that have occurred, modified based on the average branchscore for branches utilizing the particular branch as a root (e.g., eachbranch created with the particular branch as a root), and/or modifiedbased on other factors. As described herein, a particular branch is“traversed” when the particular branch is utilized in creating a branch(e.g., selected to serve as a root to generate branches), serves as theroot for a future branch (e.g., when a second branch is generated basedon the particular branch, the particular branch may be traversedmultiple times), and/or in other situations. For example, COS 101 maygenerate a branch score based on Formula 1:

$\begin{matrix}{S_{i} = {x_{i} + {C\sqrt{\frac{\ln(t)}{n_{i}}}}}} & \left( {{Formula}1} \right)\end{matrix}$In Formula 1, “S_(i)” may represent the branch score for a particularbranch, “x_(i)” may represent the average overall configuration scorebased on selection of the particular branch, “C” may represent aconstant value, such as

$\frac{1}{\sqrt{2}},$“t” may represent the number of iterations of generating branches thathave taken place, and “n_(i)” may represent the number of times theparticular branch has been selected in those iterations of generatingbranches.

In one embodiment, COS 101 may select branch 209 by traversing thehighest scoring branch in each tier as shown in FIG. 2B. As describedabove, the branch scores associated with branch 203 and branch 204 maybe modified based on the traversal of each branch. As shown, the branchscore associated with branch 203 may decrease to 28 (from 30) and thebranch score associated with branch 204 may decrease to 34 (from 35). Asshown, COS 101 may generate a new tier consisting of branch 210 (with abranch score of 52), branch 211 (with a branch score of 56), and branch212 (with a branch score of 58).

The selection of the branch is shown for explanatory purposes only. Asdiscussed below, the selection of a previous branch can occur at one ormore other branches in one or more different tiers. For example, whileshown as selecting a branch from Tier 2, selection may occur fromanother tier (Tier 3, Tier 4, etc.). In some embodiments, a branch maybe selected and a modified container list may be utilized to placecontainers in the node. In other words, COS 101 may select the samebranch as a root, but utilize a different arrangement of containers(e.g., different container placement order, etc.) to generate adifferent branch.

In some embodiments, different iterations may be generated in a randomfashion, and/or a sequential fashion (e.g., placing nodes in differentsequences, moving the first node to the end of the list in subsequentiterations). In some embodiments, different iterations may be generatedand/or scored (e.g., branch score) based on machine learning and/orother suitable techniques. For example, COS 101 may identify brancheswith relatively high branch scores (e.g., as compared to otherconfigurations with the same branches to place), and may identifycharacteristics of these branches, including resource scores of placedcontainers, sequences in which the containers are placed, resourcescores of branches on which the containers are placed, and/or sequencesof nodes on which the containers are placed. In some embodiments, COS101 may generate a ranked list of sequences of containers to placeand/or nodes on which to place them, based on identifyingcharacteristics of branches with relatively high scores. When generatingnew branches, COS 101 may, in some embodiments, generate branches in asequence that is based on this ranked list.

For example, COS 101 may identify that a set of branches and/orconfigurations, in which containers are placed in the order C1, C3, C5,C2, and then C4 (and so on), tend to have higher branch scores (e.g., anaverage, median, or other measure of the branch scores) than a set ofbranches and/or configurations in which containers are placed in theorder C5, C4, C3, C2, and then C1 (and so on). Thus, in subsequentiterations, COS 101 may generate and score branches and/orconfigurations based on the order C1, C3, C5, C2, and then C4 (e.g.,using this same order, or with variations based on this order) beforegenerating and scoring branches and/or configurations based on the orderC5, C4, C3, C2, and then C1. Additionally, or alternatively, COS 101 maygenerate and score branches and/or configurations in a weighted manner,in which COS 101 generates and scores more branches and/orconfigurations that are based on the order C1, C3, C5, C2, and then C4than branches and/or configurations that are based on the order C5, C4,C3, C2, and then C1.

As further shown in FIG. 2C, COS 101 may select branch 207 from the sametier as branch 209 to generate new branches. As described above, COS 101may select from a previous tier based on a branch score and/or for otherreasons. For example, assume that none of the branches generated basedon branch 209 (i.e., branch 210, branch 211, and/or branch 212) havesuitable container placements because the branches are restricted byanti-affinity restrictions. Therefore, COS 101 may select branch 207 togenerate additional branches. As described above, when COS 101 utilizesbranch 207 to generate additional branches, COS 101 may modify (e.g.,decrease) a branch score associated with each branch on which branch213, branch 214, and branch 215 were based (i.e., branch 203, branch204, branch 207, etc.).

As shown in FIG. 2D, as a result of the modified branch score associatedwith branch 203, COS 101 may select a different branch from that tier asa root for new branches. For example, COS 101 may select branch 202 togenerate new branches (i.e., branch 216 with a branch score of 40,branch 217 with a branch score of 42, and branch 218 with a branch scoreof 36).

In some embodiments, COS 101 may stop iterating the generation ofbranches for a particular set of nodes and/or containers based ondetermining that a limiting criterion has been reached. This limitingcriterion may indicate, for example, that branch generation should ceasewhen an optimal score has been reached. In some embodiments, an“optimal” score may be reached when, for example, the score differencebetween branches representing a complete configuration (e.g.,configuration scores, which may be associated with “leaves” of the tree)is below a threshold. In other words, if multiple configurations areassociated with a similar (or same) configuration score, COS 101 maydetermine that an “optimal” score has been reached, and may ceasegenerating configurations based on one or more branches from which theconfigurations with similar scores have been generated. As anotherexample, COS 101 may determine that an “optimal” score has been reachedwhen one or more configuration scores (and/or a score associated with abranch representing a completed configuration, such as a “leaf” of thetree) exceed a threshold value.

In some embodiments, COS 101 may continue generating branches based onone or more other limiting criteria. These limiting criteria mayinclude, for example, a limit on the amount of time spent generatingconfigurations and/or branches for a given set of containers and nodes(e.g., 20 seconds, 60 seconds, 30 minutes, etc.), a limit on thequantity of branches generated, and/or some other limiting criteria.

As described above, a configuration may be generated by identifying thehighest scoring branch in each tier, and selecting that branch as a rootfor generating the next tier. Once a configuration is generated, COS 101may (e.g., on a subsequent iteration) select a particular branch, whichwas not previously traversed in a previous iteration, to generateanother (e.g., alternate) configuration.

Once multiple configurations are generated, COS 101 may choose from thegenerated configurations, in order to fulfill a request for aconfiguration that matches (or most closely matches) a set of inputparameters (e.g., specifying parameters of containers to instantiate onone or more nodes, parameters associated with the set of nodes, one ormore types or resources to optimize and/or other criteria specifying howto rank or generate configuration scores for configurations, etc.). Forexample, as shown in FIG. 3 , COS 101 may receive several configurations(e.g., as shown configurations 305-1-305-4) from configurationrepository 303. Some or all of the configurations, stored inconfiguration repository 303, may have been generated and/or scored byCOS 101 (e.g., as similarly discussed above). As discussed above, eachconfiguration may be associated with one or more configuration scores.In some embodiments, COS 101 may rank configurations based on theirconfiguration scores and select a configuration based on the ranking(e.g., a configuration with a highest configuration score) to fulfill arequest for a configuration. As discussed below, a particularconfiguration score, out of a set of configuration scores for eachconfiguration, may be used to select a particular configuration. Forexample, the input parameters may specify that processing resourcesshould be optimized. COS 101 may thus rank configurations according toscoring of processing resources when selecting a particularconfiguration to provide in response to the request.

FIG. 4 illustrates an example of selecting and configuring a particularconfiguration, based on input configuration parameters, out of a set ofcandidate configurations. As shown in FIG. 4 , for example, COS 101 mayreceive (at 402) a set of configurations from configuration repository303. As described above, configuration repository 303 may maintaininformation regarding several configurations, including one or moreconfiguration scores associated with each configuration (e.g., asdetermined by COS 101).

As further shown in FIG. 4 , COS 101 may receive and/or determine (at404) configuration information indicating parameters of a desiredconfiguration. The parameters of the desired configuration may includeinformation regarding one or more containers to be placed in one or morenodes. In some embodiments, the configuration information may specifyattributes of the nodes in which to place the containers, such as thequantity of nodes, resources associated with the nodes, etc. In someembodiments, COS 101 may have access to a pool or set of candidate nodes407, on which one or more containers may be instantiated (and/or mayhave access to a cluster or other set of resources, on which nodes 407can be provisioned). In some embodiments, nodes 407 may be, and/or mayimplement, components of a network (e.g., a Software-Defined Network(“SDN”)), such as a wireless network (e.g., a Fifth Generation (“5G”)network). In some embodiments, COS 101 may determine a set of parametersbased on a received set of nodes and/or containers. For example, COS 101may utilize machine learning techniques to determine parameters based onsimilar sets of nodes and/or containers which were previouslyconfigured. In some embodiments, COS 101 may determine configurationinformation based on a configuration in use. In such an embodiment, COS101 may receive a configuration of nodes and containers and determineattributes associated with the nodes and/or containers. In someembodiments, COS 101 may determine configuration information based on aconfiguration in use. In such an embodiment, COS 101 may receive aconfiguration of nodes and containers and determine attributesassociated with the nodes and/or containers. For example, COS 101 mayanalyze one or more currently existing or installed configurations(e.g., determine a configuration score for an existing configuration),in order to optimize the currently existing configuration based on acorresponding configuration generated by COS 101, which may have a morefavorable (e.g., higher) configuration score than the existingconfiguration.

In some embodiments, the parameters of the desired configuration mayinclude additional configuration criteria, such as an indication that aparticular type of resource (e.g., processing resources, memoryresources, storage resources, etc.) should be optimized, an indicationthat the fewest nodes as possible should be used, an indication that thefewest amount of cumulative resources across selected nodes should beprovisioned for the configuration, and/or other criteria or combinationsof criteria.

In some embodiments, COS 101 may use a generated configuration to“resize” one or more nodes or containers (e.g., nodes and/or containersin existing configurations). “Resizing” nodes and/or containers mayinclude, for example, increasing and/or decreasing the resourcesassociated with a container and/or a node based on one or more factors.For example, containers and/or nodes may be resized based on efficiencywith a node and/or container (e.g., the resources utilized by acontainer may decrease when placed in a particular arrangement, etc.),based on scaling factors (e.g., an over-provisioning and/or anunder-provisioning of resources associated with a node and/or container,etc.), and/or some other factors. In some embodiments, COS 101 mayresize containers based on available node resources and/or a resizerange associated with a container (e.g., a limitation for an increase ordecrease in the resource utilization of a container). These limitationsmay be explicitly indicated (e.g., in metadata associated withcontainers), and/or may have been determined based on machine learningor other suitable techniques (e.g., an analysis of varying levels ofresources used for containers having one or more similar or commonattributes).

For example, assume a particular node in a configuration has an amountof a particular resource equivalent to a resource score of 5, and acontainer placed in the particular node causes the remaining resourcescore for this particular resource to be reduced to 1 (e.g., consumes anamount of this particular resource equivalent to a resource score of 4).In another manner of considering this example, the particular node mayhave 5 Terabytes (“TB”) of total storage space, and a particularcontainer placed on the node (according to a given configuration) mayconsume 4 TB of storage space, leaving 1 TB of storage space remainingon the node. In such an instance, COS 101 may resize the container toutilize the additional 1 TB of storage space when placing the containerin the particular node. For example, COS 101 may modify the container toutilize 5 TB of storage space, instead of 4 TB.

Similarly, in some situations, the resources associated with a containermay be decreased. Assume that a particular container is associated witha range and/or minimum value for a given resource. For example, theparticular container may be associated with a storage space of 4 TB, butmay also be associated with a minimum storage space of 2 TB. Referringagain to the example of this container being placed on a node with 5 TBof total storage space, COS 101 may reduce the storage space associatedwith the container, so that the remaining storage space associated withthe node is greater than 1 TB (i.e., where 1 TB would be left over ifthe container, utilizing 4 TB of storage space, were placed on the nodewith 5 TB of total storage space).

COS 101 may, in some aspects, utilize machine learning and/or othertechniques to determine the amount to reduce the container's resourcesby, and/or to determine the amount of resources to leave available afterresizing. For instance, COS 101 may analyze a set of configurations(e.g., as generated by COS 101, and/or existing or currently installedconfigurations) to determine an amount of resources that are likely tobe useful or usable. Based on the analysis of configurations, COS 101may determine that a remaining storage space of 2.7 TB would be likelyto be usable by another container (e.g., containers that are likely tobe installed on the node may consume or require up to 2.7 TB of storagespace). This analysis may include analyzing other configurations todetermine particular containers placed on the nodes in theseconfigurations, and determining other containers that are included inthe same node and/or configuration as the particular containers. Assuch, based on this determination, COS 101 may resize the container fromutilizing 4 TB of storage space to 2.3 TB of storage space. Thisresizing would result in the container utilizing at least its minimumstorage space of 2 TB, and would also result in the node having 2.7 TBof storage space available (e.g., available for the installation of oneor more other containers).

As further shown in FIG. 4 , COS 101 may select (at 406) a configurationfrom the provided configurations. COS 101 may compare the desiredconfiguration information, including any criteria, to the providedconfiguration. COS 101 may rank each configuration based on theassociated configuration score and/or similarity to the criteria (e.g.,based on a suitable similarity analysis). In some embodiments, COS 101may reject (i.e., not select from) a configuration which does notsatisfy the desired configuration information. In some embodiments, theselection may be modified based on machine learning techniques. Machinelearning techniques may enhance selection by analyzing feedback todetermine a confidence level associated with a selection. For example,COS 101 may maintain and/or refine predictive models associated witheach configuration (e.g., to strength or weaken a correlation between agiven configuration and the received configuration information).Predictive models may contain, for example, information regardingprevious selection of the configuration (e.g., the configurationinformation used to previously select a configuration). Feedback may beused to modify the predictive models. For example, feedback indicating acorrect selection based on the same/similar information may strengthenor increase the association of the selected configuration with thecriteria or parameters used to select the configuration, whereasfeedback indicating an incorrect selection based on the same/similarinformation may weaken or decrease this association. As described below,different feedback may modify the predictive model score differently(e.g., higher confidence feedback may impact a predictive model scoremore than lower confidence feedback).

As described above, selection information (e.g., received at 404) mayinclude additional criteria to select a configuration. For example,selection information may include an indication to utilize the fewestnumber of nodes (e.g., optimize the number of nodes utilized). Asdescribed above, each node may represent a discrete set of hardwareresources. The optimization of the number of nodes may reduce thehardware requirements for the placement of containers. Someconfigurations may utilize fewer nodes compared to other configurationswhen utilizing the same nodes and/or containers.

In some embodiments, selection information may include an indication tooptimize a particular resource. As described above, one or moreresources may be identified and/or scored for each placed node and/orcontainer. For the sake of example, assume an indication to optimizebased on a first resource type, and a first node has a high startingresource score for the first resource type (indicating a high amount ofthe first resource) and a second node has a low starting resource scorefor the first resource type (indicating a lower amount of the firstresource compared to the first node). Assume further that all containersto be placed may be placed in either the first node or the second node.A configuration utilizing the second node to place all containers may beselected over a configuration utilizing the first node to place allcontainers because the second node may optimize the first resource type.In other words, a configuration utilizing the second node would leavefewer unused resources of the first resource type. In some embodiments,configuration information may include an indication to optimize morethan one resource in order. Assume for the sake of example that threeresource types (e.g., Resource 1, Resource 2, Resource 3) are scoredand/or identified. The received configuration information may indicateto optimize Resource 1, then Resource 3, and then Resource 2. Selectionmay occur based on this prioritization for resource optimization. Thisprioritization may occur, for example, in embodiments where multiplenodes have high starting resource scores.

In some embodiments, configuration information may indicate theselection of a configuration utilizing the fewest amount of cumulativeresources across all nodes in the configuration. This selection criteriamay, for example, indicate a selection of a configuration wherein thenodes are full (e.g., the resources associated with each node in theconfiguration are utilized). In some embodiments, a selection based onthis configuration information may utilize many nodes with relativelysmall resource capacity rather than fewer relatively large resourcecapacity nodes. For the sake of example, assume a first configurationutilizes a single node with a resource score of 20 to place a set ofcontainers using 13 resources, and a second configuration utilizes twonodes with a total resource score of 14 to place the same set ofcontainers using 13 resources. Given an indication to select aconfiguration based on utilizing the fewest amount of cumulativeresources, COS 101 may select the second configuration (i.e., two nodeswith a total resource score of 14) despite utilizing two nodes ratherthan one.

In some embodiments, COS 101 may select a configuration based on aconfiguration currently in use. For example, COS 101 may analyze apreviously configured set of nodes and containers to determine amismatch with the highest scoring configuration. In such embodiments,COS 101 may select the highest scoring configuration based on theresources associated with nodes and/or containers and/or otherparameters. In other words, COS 101 may rebalance a currentconfiguration to better utilize resources in accordance with selectparameters.

As further shown in FIG. 4 , COS 101 may configure (e.g., install,instantiate, etc.) (at 408) the selected configuration onto node(s) 407(e.g., each node). COS 101 may configure the selected configuration byutilizing an API (e.g., Kubernetes API, etc.). In some embodiments, COS101 may be communicatively coupled with another device, such as anorchestration system, which may instantiate the containers of theconfiguration onto the node and/or may provision the nodes from acluster of available resources according to the selected configuration.In such embodiments, COS 101 may provide instructions to anorchestration system on how to configure the nodes and/or the cluster ofavailable resources.

COS 101 may receive (at 410) feedback regarding the configuredconfiguration (e.g., from 410). Feedback may be utilized to enhancefuture selection via machine learning techniques based on the providedfeedback, the confidence associated with the feedback, and/or otherfactors. For example, in some embodiments, a user associated with COS101 may select a different configuration and/or provide feedbackregarding the configuration selection after COS 101 selected (e.g., at406) a configuration based on provided configuration information (e.g.,from 404). For example, a user associated with COS 101 may select adifferent configuration than a configuration selected by COS 101. Thisuser selection may indicate that the user-selected configuration is moresuitable than the configuration selected by COS 101 (e.g., theuser-selected configuration better matches the provided configurationinformation). In some embodiments, a user may modify a selectedconfiguration (e.g., a user may switch, remove, replace, and/orotherwise modify nodes and/or containers in a configuration). Such amodification may indicate that the modified configuration is moresuitable than the unmodified configuration selected by COS 101. Modifiedand/or user-selected configurations may be configured onto nodes (e.g.,as described above, in regard to 410). In such embodiments, where a usermodifies and/or selects a different configuration, COS 101 may storeinformation regarding the configuration selection to provideuser-reinforced feedback regarding the selection. User-reinforcedfeedback may be used to enhance future configuration selection with ahigh level of confidence. Different levels of confidence may indicate,for example, the degree to which a predictive model is modified.Feedback with a relatively high degree of confidence, for example, mayimpact a predictive model more heavily than feedback with relatively alow degree of confidence.

In some embodiments, a user may provide feedback affirming the selectionof a particular configuration. The feedback may include explicitaffirmative feedback, such as an affirmative answer when promptedwhether the selection was accurate or satisfactory. In some embodiments,the feedback may include one or more actions based on which feedback maybe inferred, such as proceeding to use and/or not change the selectedconfiguration within a threshold period of time (e.g., indicating thatthe user was satisfied with the selection and therefore did no changethe configuration). When receiving feedback affirming the selection of aparticular configuration, COS 101 may strengthen predictive modelsassociated with that selection. In other words, given the same and/orsimilar configuration information, COS 101 would be more likely tochoose the same configuration. If a user provides a selection rejectingthe selection made by COS 101 (e.g., an explicit indication that theselection was not satisfactory, such as a response to a prompt asking ifthe selection was satisfactory or accurate, or an action based on whichfeedback may be inferred, such as modifying the selection within athreshold period of time), COS 101 may be less likely to choose aconfiguration given the same and/or similar configuration information.

FIG. 5 illustrates an example environment 500, in which one or moreembodiments may be implemented. In some embodiments, environment 500 maycorrespond to a Fifth Generation (“5G”) network, and/or may includeelements of a 5G network. In some embodiments, environment 500 maycorrespond to a 5G Non-Standalone (“NSA”) architecture, in which a 5Gradio access technology (“RAT”) may be used in conjunction with one ormore other RATs (e.g., a Long-Term Evolution (“LTE”) RAT), and/or inwhich elements of a 5G core network may be implemented by, may becommunicatively coupled with, and/or may include elements of anothertype of core network (e.g., an evolved packet core (“EPC”)). As shown,environment 500 may include UE 501, radio access network (“RAN”) 510(which may include one or more Next Generation Node Bs (“gNBs”) 511),RAN 512 (which may include one or more one or more evolved Node Bs(“eNBs”) 513), AMF 515, SMF/PGW-Control plane function (“PGW-C”) 520,PCF/Policy Charging and Rules Function (“PCRF”) 525, AF 530,UPF/PGW-User plane function (“PGW-U”) 535, HSS/UDM 540, AUSF 545, DataNetwork (“DN”) 550, and COS 101.

The quantity of devices and/or networks, illustrated in FIG. 5 , isprovided for explanatory purposes only. In practice, environment 500 mayinclude additional devices and/or networks, fewer devices and/ornetworks, different devices and/or networks, or differently arrangeddevices and/or networks than illustrated in FIG. 5 . For example, whilenot shown, environment 500 may include devices that facilitate or enablecommunication between various components shown in environment 500, suchas routers, modems, gateways, switches, hubs, etc. Alternatively, oradditionally, one or more of the devices of environment 500 may performone or more functions described as being performed by another one ormore of the devices of environment 500. Devices of environment 500 mayinterconnect with each other and/or other devices via wired connections,wireless connections, or a combination of wired and wirelessconnections. In some implementations, one or more devices of environment500 may be physically integrated in, and/or may be physically attachedto, one or more other devices of environment 500.

UE 501 may include a computation and communication device, such as awireless mobile communication device that is capable of communicatingwith RAN 510 and/or DN 550. UE 501 may be, or may include, aradiotelephone, a personal communications system (“PCS”) terminal (e.g.,a device that combines a cellular radiotelephone with data processingand data communications capabilities), a personal digital assistant(“PDA”) (e.g., a device that may include a radiotelephone, a pager,Internet/intranet access, etc.), a smart phone, a laptop computer, atablet computer, a camera, a personal gaming system, an IoT device(e.g., a sensor, a smart home appliance, or the like), a wearabledevice, a Mobile-to-Mobile (“M2M”) device, or another type of mobilecomputation and communication device. UE 501 may send traffic to and/orreceive traffic (e.g., user plane traffic) from DN 550 via RAN 510 andUPF/PGW-U 535.

RAN 510 may be, or may include, a 5G RAN that includes one or more basestations (e.g., one or more gNBs 511), via which UE 501 may communicatewith one or more other elements of environment 500. UE 501 maycommunicate with RAN 510 via an air interface (e.g., as provided by gNB511). For instance, RAN 510 may receive traffic (e.g., voice calltraffic, data traffic, messaging traffic, signaling traffic, etc.) fromUE 501 via the air interface, and may communicate the traffic toUPF/PGW-U 535, and/or one or more other devices or networks. Similarly,RAN 510 may receive traffic intended for UE 501 (e.g., from UPF/PGW-U535, AMF 515, and/or one or more other devices or networks) and maycommunicate the traffic to UE 501 via the air interface.

RAN 512 may be, or may include, an LTE RAN that includes one or morebase stations (e.g., one or more eNBs 513), via which UE 501 maycommunicate with one or more other elements of environment 500. UE 501may communicate with RAN 512 via an air interface (e.g., as provided byeNB 513). For instance, RAN 510 may receive traffic (e.g., voice calltraffic, data traffic, messaging traffic, signaling traffic, etc.) fromUE 501 via the air interface, and may communicate the traffic toUPF/PGW-U 535, and/or one or more other devices or networks. Similarly,RAN 510 may receive traffic intended for UE 501 (e.g., from UPF/PGW-U535, SGW 517, and/or one or more other devices or networks) and maycommunicate the traffic to UE 501 via the air interface.

AMF 515 may include one or more devices, systems, Virtualized NetworkFunctions (“VNFs”), etc., that perform operations to register UE 501with the 5G network, to establish bearer channels associated with asession with UE 501, to hand off UE 501 from the 5G network to anothernetwork, to hand off UE 501 from the other network to the 5G network,and/or to perform other operations. In some embodiments, the 5G networkmay include multiple AMFs 515, which communicate with each other via theN14 interface (denoted in FIG. 5 by the line marked “N14” originatingand terminating at AMF 515).

SGW 517 may include one or more devices, systems, VNFs, etc., thataggregate traffic received from one or more eNBs 513 and send theaggregated traffic to an external network or device via UPF/PGW-U 535.Additionally, SGW 517 may aggregate traffic received from one or moreUPF/PGW-Us 535 and may send the aggregated traffic to one or more eNBs513. SGW 517 may operate as an anchor for the user plane duringinter-eNB handovers and as an anchor for mobility between differenttelecommunication networks or RANs (e.g., RANs 510 and 512).

SMF/PGW-C 520 may include one or more devices, systems, VNFs, etc., thatgather, process, store, and/or provide information in a manner describedherein. SMF/PGW-C 520 may, for example, facilitate in the establishmentof communication sessions on behalf of UE 501. In some embodiments, theestablishment of communications sessions may be performed in accordancewith one or more policies provided by PCF/PCRF 525.

PCF/PCRF 525 may include one or more devices, systems, VNFs, etc., thataggregate information to and from the 5G network and/or other sources.PCF/PCRF 525 may receive information regarding policies and/orsubscriptions from one or more sources, such as subscriber databasesand/or from one or more users (such as, for example, an administratorassociated with PCF/PCRF 525).

AF 530 may include one or more devices, systems, VNFs, etc., thatreceive, store, and/or provide information that may be used indetermining parameters (e.g., quality of service parameters, chargingparameters, or the like) for certain applications.

UPF/PGW-U 535 may include one or more devices, systems, VNFs, etc., thatreceive, store, and/or provide data (e.g., user plane data). Forexample, UPF/PGW-U 535 may receive user plane data (e.g., voice calltraffic, data traffic, etc.), destined for UE 501, from DN 550, and mayforward the user plane data toward UE 501 (e.g., via RAN 510, SMF/PGW-C520, and/or one or more other devices). In some embodiments, multipleUPFs 535 may be deployed (e.g., in different geographical locations),and the delivery of content to UE 501 may be coordinated via the N9interface (e.g., as denoted in FIG. 5 by the line marked “N9”originating and terminating at UPF/PGW-U 535). Similarly, UPF/PGW-U 535may receive traffic from UE 501 (e.g., via RAN 510, SMF/PGW-C 520,and/or one or more other devices), and may forward the traffic toward DN550. In some embodiments, UPF/PGW-U 535 may communicate (e.g., via theN4 interface) with SMF/PGW-C 520, regarding user plane data processed byUPF/PGW-U 535.

HSS/UDM 540 and AUSF 545 may include one or more devices, systems, VNFs,etc., that manage, update, and/or store, in one or more memory devicesassociated with AUSF 545 and/or HSS/UDM 540, profile informationassociated with a subscriber. AUSF 545 and/or HSS/UDM 540 may performauthentication, authorization, and/or accounting operations associatedwith the subscriber and/or a communication session with UE 501.

DN 550 may include one or more wired and/or wireless networks. Forexample, DN 550 may include an Internet Protocol (“IP”)-based PDN, awide area network (“WAN”) such as the Internet, a private enterprisenetwork, and/or one or more other networks. UE 501 may communicate,through DN 550, with data servers, other UEs 501, and/or to otherservers or applications that are coupled to DN 550. DN 550 may beconnected to one or more other networks, such as a public switchedtelephone network (“PSTN”), a public land mobile network (“PLMN”),and/or another network. DN 550 may be connected to one or more devices,such as content providers, applications, web servers, and/or otherdevices, with which UE 501 may communicate.

COS 101 may include one or more devices or systems that perform one ormore operations described herein. For example, COS 101 may communicatewith one or more of the devices, systems, VNFs, or networks shown inFIG. 5 , may generate configurations (e.g., using machine learningand/or other techniques), and may configure, install, instantiate, etc.nodes and/or containers in accordance with a selected configuration. Asmentioned above, COS 101 may include, or may be communicatively coupledto, an orchestration system that is capable of receiving information(e.g., diagnostic or monitoring information) from the devices or systemsshown in FIG. 5 , and is capable of installing or instantiatingcontainers on nodes (e.g., as implemented by physical or virtualhardware). While no explicit connections are shown between COS 101 andother devices or systems of environment 500, COS 101 may, in someembodiments, be communicatively coupled to some or all of the devices orsystems of environment 500. In some embodiments, the same devices,nodes, clusters, etc., that implement one or more of the devices orsystems of environment 500 may implement some or all of thefunctionality of COS 101.

FIG. 6 illustrates an example process 600 for generating configurations,in accordance with some embodiments. In some embodiments, some or all ofprocess 600 may be performed by COS 101. In some embodiments, one ormore other devices may perform some or all of process 600 (e.g., inconcert with, and/or in lieu of, COS 101).

As shown, process 600 may include determining (at 602) container andnode parameters for configuration generation. As described above, suchparameters may include information regarding the nodes and/orcontainers, such as resources associated with containers and/or nodesand/or other information. As described above, container and nodeparameters may be based on current applications in use and/or may begenerated based on predicted future use. In some embodiments,configuration parameters may be generated (e.g., using random resourceinformation, based on predicted usage, etc.) for configuration creation.

Process 600 may further include selecting (at 604) a root to generateone or more branches. As described above, a branch may serve as a rootto generate one or more additional branches. In some embodiments, anempty node may serve as a root for branch generation. As describedabove, a root may be selected based on a branch score (e.g., the highestscoring branch) and/or other criteria. In some embodiments, a node maybe selected based on a node order and/or other criteria.

Process 600 may further include simulating (at 606) branches based onthe selected base branch. As described above, simulating branches mayinclude generating one or more branches by placing one or morecontainers into the selected base branch.

Process 600 may also include determining (at 608) scoring metricsassociated with the generated (or simulated) branches. Scoring metricsmay include criteria by which to score the branches. As described above,for example, a branch may be scored (e.g., to generate a branch score)based on the number of resources utilized (e.g., resources remainingand/or used, etc.), and/or other scoring factors.

Process 600 may also include scoring (at 610) branches. Scoring mayoccur based on determined scoring metrics (e.g., from 608). Scoring mayoccur, for example based on a numerical rating, a classification rating(e.g., grade A, grade B, etc.), and/or other ratings. COS 101 may, insome embodiments, generate multiple branch scores for each branch, whereeach branch score is based on different scoring metrics. The differentbranch scores, according to the different scoring metrics, may be usedin the selection of a particular branch to provide in response to arequest that specifies one or more parameters to be prioritized. Forexample, a request that specifies optimization of processor resourcesmay cause COS 101 to identify a particular branch that has a highestbranch score that was generated based on processor resources utilized.As discussed above, in some embodiments, one or more branch scoresassociated with a branch may be modified based on utilization togenerate branches (e.g., branch scores may be decreased when traversed).

Process 600 may further include storing (at 612) the branch with relatedbranch scores. In some embodiments, the configuration may be stored withthe related branch score, in a repository (e.g., configurationrepository 303) and/or some other data structure capable of maintainingnode information. In some embodiments, the storing may further providedetails regarding the configuration information utilized to simulate thebranch. In some embodiments, storing the branch score may includemaintaining one or more branch scores associated with a previouslygenerated branch.

Process 600 may further include a determination (at 614) whether theiterations are complete. The iterations may be complete, for example,based on received configuration information such as parameters limitingthe simulation to a number of iterations, a specific time in which tocomplete iterations (e.g., a time limit), etc. If the determinationindicates that the iterations are not complete, COS 101 may repeat602-614 to generate one or more iterations of branches before selectingbranches to form a configuration.

As shown, if the determination (at 614) is that the iterations arecomplete, COS 101 may determine (at 616) a configuration based onsimulated branches. As described above, COS 101 may select the highestscoring branches associated with each node to generate a configuration.

Process 600 may further include scoring (at 618) a configuration basedon scoring metrics. Scoring may occur based on various criteria. In someembodiments, a configuration score may be generated for theconfiguration in accordance with the determined scoring metrics (e.g.,based on the same scoring metrics used to score branches, from 608). COS101 may, in some embodiments, generate multiple configuration scores foreach configuration, where each configuration score is based on differentscoring metrics. The different configuration scores, according to thedifferent scoring metrics, may be used in the selection of a particularconfiguration to provide in response to a request that specifies one ormore parameters to be prioritized. For example, a request that specifiesoptimization of processor resources may cause COS 101 to identify aparticular configuration based on the configuration scores (e.g., aconfiguration that has a highest configuration score) that weregenerated based on processor resources utilized.

Process 600 may further include storing (at 620) a configuration withthe related configuration score. In some embodiments, the configurationmay be stored with related configuration score(s) in a repository (e.g.,configuration repository 303) and/or some other data structure capableof maintaining configuration information. In some embodiments, theprovision may further provide details regarding the configurationinformation utilized to create the configuration.

Once information regarding a configuration is stored, portions ofprocess 600 (e.g., some or all of blocks 602-620) may iteratively repeatin order to generate one or more configurations, which may use the sameor different parameters (e.g., as received at 602). In some embodiments,configuration information may limit the number of configurations togenerate based on a configuration limitation (e.g., limit the number ofconfigurations to generate), timing (e.g., generate for a limited timeperiod), and/or other restrictions.

FIG. 7 illustrates an example process of selecting and configuring aconfiguration in response to a request for a configuration. Process 700may generally correspond, for example, to a situation in which a set ofcontainers are to be placed into a set of nodes. For example, a user maydesire to place the set of containers onto a predetermined set of nodes,and/or onto an optimal set of nodes as indicated by a particularconfiguration that has been generated by COS 101. In some embodiments,some or all of process 700 may be performed by COS 101. In someembodiments, one or more other devices may perform some or all ofprocess 700 (e.g., in concert with, and/or in lieu of, COS 101).

Process 700 may include receiving (at 702) selection information. Asdescribed above, selection information may include container and/or nodeinformation. In some embodiments, selection information may includeadditional criteria (e.g., optimize a particular resource, optimizenumber of nodes, etc.).

Process 700 may additionally include selecting (at 704) a configuration.As described above, COS 101 may receive a set of configurations (e.g.,from configuration repository 303) and one or more configuration scoresassociated with each configuration of the set of configurations. Asdescribed above, a configuration may be chosen based on an associatedconfiguration score. For example, a configuration may be selectedbecause the configuration has the highest configuration score for theselection parameters (e.g., received at 702). In some embodiments, asimilarity analysis may be performed between the selection information(received at 702) and the one or more configurations stored inconfiguration repository 303, in order to identify a particularconfiguration that has a highest degree of similarity to the selectioninformation. As discussed above, one or more machine learning techniquesmay have been used to correlate attributes of selection information toattributes of one or more configurations.

Process 700 may also include configuring (at 706) a node in accordancewith the selected configuration. As described above, COS 101 mayconfigure (e.g., install, instantiate, etc.) and/or may provideinstructions to another device to configure nodes and/or containers.

Process 700 may include receiving (at 708) feedback regarding theconfiguration. As discussed above, feedback may enhance futureconfiguration selection based on previous use (e.g., utilizing machinelearning techniques). In some embodiments, explicit feedback may bereceived from a user (e.g., providing relatively high degree ofconfidence, as discussed above). In some embodiments, feedback may beinferred by the absence of explicit feedback from a user (e.g.,providing relatively low degree of confidence, as discussed above).

Process 700 may further include refining (at 710) one or more modelsthat were used to select the particular configuration for the selectioninformation. For example, based on the feedback, COS 101 may determinewhether the selection (at 704) of the particular configuration wasaccurate or otherwise appropriate for the selection information(received at 702). COS 101 may refine the association or correlation ofsome or all of the selection information to the particular configurationthat was selected, based on the feedback (received at 708).

FIG. 8 illustrates example components of device 800. One or more of thedevices described above may include one or more devices 800. Device 800may include bus 810, processor 820, memory 830, input component 840,output component 850, and communication interface 860. In anotherimplementation, device 800 may include additional, fewer, different, ordifferently arranged components.

Bus 810 may include one or more communication paths that permitcommunication among the components of device 800. Processor 820 mayinclude a processor, microprocessor, or processing logic that mayinterpret and execute instructions. Memory 830 may include any type ofdynamic storage device that may store information and instructions forexecution by processor 820, and/or any type of non-volatile storagedevice that may store information for use by processor 820.

Input component 840 may include a mechanism that permits an operator toinput information to device 800, such as a keyboard, a keypad, a button,a switch, etc. Output component 850 may include a mechanism that outputsinformation to the operator, such as a display, a speaker, one or morelight emitting diodes (“LEDs”), etc.

Communication interface 860 may include any transceiver-like mechanismthat enables device 800 to communicate with other devices and/orsystems. For example, communication interface 860 may include anEthernet interface, an optical interface, a coaxial interface, or thelike. Communication interface 860 may include a wireless communicationdevice, such as an infrared (“IR”) receiver, a Bluetooth® radio, or thelike. The wireless communication device may be coupled to an externaldevice, such as a remote control, a wireless keyboard, a mobiletelephone, etc. In some embodiments, device 800 may include more thanone communication interface 860. For instance, device 800 may include anoptical interface and an Ethernet interface.

Device 800 may perform certain operations relating to one or moreprocesses described above. Device 800 may perform these operations inresponse to processor 820 executing software instructions stored in acomputer-readable medium, such as memory 830. A computer-readable mediummay be defined as a non-transitory memory device. A memory device mayinclude space within a single physical memory device or spread acrossmultiple physical memory devices. The software instructions may be readinto memory 830 from another computer-readable medium or from anotherdevice. The software instructions stored in memory 830 may causeprocessor 820 to perform processes described herein. Alternatively,hardwired circuitry may be used in place of or in combination withsoftware instructions to implement processes described herein. Thus,implementations described herein are not limited to any specificcombination of hardware circuitry and software.

The foregoing description of implementations provides illustration anddescription, but is not intended to be exhaustive or to limit thepossible implementations to the precise form disclosed. Modificationsand variations are possible in light of the above disclosure or may beacquired from practice of the implementations.

For example, while series of blocks and/or signals have been describedabove (e.g., with regard to FIGS. 1A-1H, 2A-2D, 4, 6, and 7 ), the orderof the blocks and/or signals may be modified in other implementations.Further, non-dependent blocks and/or signals may be performed inparallel. Additionally, while the figures have been described in thecontext of particular devices performing particular acts, in practice,one or more other devices may perform some or all of these acts in lieuof, or in addition to, the above-mentioned devices.

The actual software code or specialized control hardware used toimplement an embodiment is not limiting of the embodiment. Thus, theoperation and behavior of the embodiment has been described withoutreference to the specific software code, it being understood thatsoftware and control hardware may be designed based on the descriptionherein.

Even though particular combinations of features are recited in theclaims and/or disclosed in the specification, these combinations are notintended to limit the disclosure of the possible implementations. Infact, many of these features may be combined in ways not specificallyrecited in the claims and/or disclosed in the specification. Althougheach dependent claim listed below may directly depend on only one otherclaim, the disclosure of the possible implementations includes eachdependent claim in combination with every other claim in the claim set.

Further, while certain connections or devices are shown, in practice,additional, fewer, or different, connections or devices may be used.Furthermore, while various devices and networks are shown separately, inpractice, the functionality of multiple devices may be performed by asingle device, or the functionality of one device may be performed bymultiple devices. Further, multiple ones of the illustrated networks maybe included in a single network, or a particular network may includemultiple networks. Further, while some devices are shown ascommunicating with a network, some such devices may be incorporated, inwhole or in part, as a part of the network.

To the extent the aforementioned implementations collect, store, oremploy personal information provided by individuals, it should beunderstood that such information shall be collected, stored, and used inaccordance with all applicable laws concerning protection of personalinformation. Additionally, the collection, storage, and use of suchinformation may be subject to consent of the individual to such activity(for example, through “opt-in” or “opt-out” processes, as may beappropriate for the situation and type of information). Storage and useof personal information may be in an appropriately secure mannerreflective of the type of information, for example, through variousencryption and anonymization techniques for particularly sensitiveinformation.

No element, act, or instruction used in the present application shouldbe construed as critical or essential unless explicitly described assuch. An instance of the use of the term “and,” as used herein, does notnecessarily preclude the interpretation that the phrase “and/or” wasintended in that instance. Similarly, an instance of the use of the term“or,” as used herein, does not necessarily preclude the interpretationthat the phrase “and/or” was intended in that instance. Also, as usedherein, the article “a” is intended to include one or more items, andmay be used interchangeably with the phrase “one or more.” Where onlyone item is intended, the terms “one,” “single,” “only,” or similarlanguage is used. Further, the phrase “based on” is intended to mean“based, at least in part, on” unless explicitly stated otherwise.

What is claimed is:
 1. A device, comprising: one or more processorsconfigured to: identify a plurality of containers that are eachassociated with a respective measure of resource utilization;iteratively simulate a plurality of different configurations of theplurality of containers on a plurality of sets of hardware resources,wherein iteratively simulating the plurality of different configurationsincludes: simulating placement of a first container and a secondcontainer, of the plurality of containers, on a first set of hardwareresources of the plurality of hardware resources, and simulatingplacement of the first container on the first set of hardware resourcesand simulating placement of the second container on a second set ofhardware resources of the plurality of hardware resources; generate ascore for each respective configuration, of the plurality of differentconfigurations, based on the measures of resource utilization associatedwith the plurality of containers; select a particular configurationbased on the generated scores; and based on the selected configuration:place the first set of containers on the first set of hardwareresources, and place the second set of containers on the second set ofhardware resources.
 2. The device of claim 1, wherein a higher score isassociated with a greater measure of resource utilization of theplurality of sets of hardware resources.
 3. The device of claim 1,wherein generating the scores includes: generating a first score for afirst configuration that is based on the simulated placement of thefirst container and the second container on the first set of hardwareresources; and generating a second score for a second configuration thatis based on the simulated placement of the first container on the firstset of hardware resources and of the second container on the second setof hardware resources.
 4. The device of claim 3, wherein the one or moreprocessors are further configured to: compare the first and secondscores, wherein the selected particular configuration is a particularone of the first configuration or the second configuration selectedbased on the comparing.
 5. A device, comprising: one or more processorsconfigured to: identify a plurality of containers that are eachassociated with a respective measure of resource utilization;iteratively simulate a plurality of different configurations of theplurality of containers on a plurality of sets of hardware resources,wherein a first configuration, of the of the simulated plurality ofdifferent configurations, includes a first quantity of sets of hardwareresources, and wherein a second configuration, of the of the simulatedplurality of different configurations, includes a second quantity ofsets of hardware resources; generate a score for each respectiveconfiguration, of the plurality of different configurations, based onthe measures of resource utilization associated with the plurality ofcontainers; compare the first quantity to the second quantity todetermine that the first configuration includes a lesser quantity ofsets of hardware resources than the second configuration; select aparticular configuration based on the generated scores, whereinselecting the particular configuration includes selecting the firstconfiguration based on determining that the first configuration includesthe lesser quantity of sets of hardware resources than the secondconfiguration, and based on the selected configuration: place a firstset of containers, of the plurality of containers, on a first set ofhardware resources of the plurality of sets of hardware resources, andplace a second set of containers, of the plurality of containers, on asecond set of hardware resources of the plurality of sets of hardwareresources.
 6. The device of claim 5, wherein the plurality of sets ofhardware resources each include a respective node of a virtualizedenvironment.
 7. The device of claim 5, wherein a higher score isassociated with a greater measure of resource utilization of theplurality of sets of hardware resources.
 8. A non-transitorycomputer-readable medium, storing a plurality of processor-executableinstructions to: identify a plurality of containers that are eachassociated with a respective measure of resource utilization;iteratively simulate a plurality of different configurations of theplurality of containers on a plurality of sets of hardware resources,wherein iteratively simulating the plurality of different configurationsincludes: simulating placement of a first container and a secondcontainer, of the plurality of containers, on a first set of hardwareresources of the plurality of hardware resources, and simulatingplacement of the first container on the first set of hardware resourcesand simulating placement of the second container on a second set ofhardware resources of the plurality of hardware resources; generate ascore for each respective configuration, of the plurality of differentconfigurations, based on the measures of resource utilization associatedwith the plurality of containers; select a particular configurationbased on the generated scores; and based on the selected configuration:place the first set of containers on the first set of hardwareresources, and place the second set of containers on the second set ofhardware resources.
 9. The non-transitory computer-readable medium ofclaim 8, wherein a higher score is associated with a greater measure ofresource utilization of the plurality of sets of hardware resources. 10.The non-transitory computer-readable medium of claim 8, whereingenerating the scores includes: generating a first score for a firstconfiguration that is based on the simulated placement of the firstcontainer and the second container on the first set of hardwareresources; and generating a second score for a second configuration thatis based on the simulated placement of the first container on the firstset of hardware resources and of the second container on the second setof hardware resources.
 11. The non-transitory computer-readable mediumof claim 10, wherein the plurality of processor-executable instructionsfurther include processor-executable instructions to: compare the firstand second scores, wherein the selected particular configuration is aparticular one of the first configuration or the second configurationselected based on the comparing.
 12. A non-transitory computer-readablemedium, storing a plurality of processor-executable instructions to:identify a plurality of containers that are each associated with arespective measure of resource utilization; iteratively simulate aplurality of different configurations of the plurality of containers ona plurality of sets of hardware resources, wherein a firstconfiguration, of the of the simulated plurality of differentconfigurations, includes a first quantity of sets of hardware resources,and wherein a second configuration, of the of the simulated plurality ofdifferent configurations, includes a second quantity of sets of hardwareresources; compare the first quantity to the second quantity todetermine that the first configuration includes a lesser quantity ofsets of hardware resources than the second configuration; generate ascore for each respective configuration, of the plurality of differentconfigurations, based on the measures of resource utilization associatedwith the plurality of containers; select a particular configurationbased on the generated scores, wherein selecting the particularconfiguration includes selecting the first configuration based ondetermining that the first configuration includes the lesser quantity ofsets of hardware resources than the second configuration; and based onthe selected configuration: place a first set of containers, of theplurality of containers, on a first set of hardware resources of theplurality of sets of hardware resources, and place a second set ofcontainers, of the plurality of containers, on a second set of hardwareresources of the plurality of sets of hardware resources.
 13. Thenon-transitory computer-readable medium of claim 12, wherein theplurality of sets of hardware resources each include a respective nodeof a virtualized environment.
 14. The non-transitory computer-readablemedium of claim 12, wherein a higher score is associated with a greatermeasure of resource utilization of the plurality of sets of hardwareresources.
 15. A method, comprising: identifying a plurality ofcontainers that are each associated with a respective measure ofresource utilization; iteratively simulating a plurality of differentconfigurations of the plurality of containers on a plurality of sets ofhardware resources, wherein iteratively simulating the plurality ofdifferent configurations includes: simulating placement of a firstcontainer and a second container, of the plurality of containers, on afirst set of hardware resources of the plurality of hardware resources,and simulating placement of the first container on the first set ofhardware resources and simulating placement of the second container on asecond set of hardware resources of the plurality of hardware resources;generating a score for each respective configuration, of the pluralityof different configurations, based on the measures of resourceutilization associated with the plurality of containers; selecting aparticular configuration based on the generated scores; and based on theselected configuration: placing the first set of containers on the firstset of hardware resources, and placing the second set of containers onthe second set of hardware resources.
 16. The method of claim 15,wherein a higher score is associated with a greater measure of resourceutilization of the plurality of sets of hardware resources.
 17. Themethod of claim 15, wherein generating the scores includes: generating afirst score for a first configuration that is based on the simulatedplacement of the first container and the second container on the firstset of hardware resources; generating a second score for a secondconfiguration that is based on the simulated placement of the firstcontainer on the first set of hardware resources and of the secondcontainer on the second set of hardware resources; and comparing thefirst and second scores, wherein the selected particular configurationis a particular one of the first configuration or the secondconfiguration selected based on the comparing.
 18. A method, comprising:identifying a plurality of containers that are each associated with arespective measure of resource utilization iteratively simulating aplurality of different configurations of the plurality of containers ona plurality of sets of hardware resources, wherein a firstconfiguration, of the of the simulated plurality of differentconfigurations, includes a first quantity of sets of hardware resources,and wherein a second configuration, of the of the simulated plurality ofdifferent configurations, includes a second quantity of sets of hardwareresources; generating a score for each respective configuration, of theplurality of different configurations, based on the measures of resourceutilization associated with the plurality of containers; comparing thefirst quantity to the second quantity to determine that the firstconfiguration includes a lesser quantity of sets of hardware resourcesthan the second configuration; selecting a particular configurationbased on the generated scores, wherein selecting the particularconfiguration includes selecting the first configuration based ondetermining that the first configuration includes the lesser quantity ofsets of hardware resources than the second configuration; and based onthe selected configuration: placing a first set of containers, of theplurality of containers, on a first set of hardware resources of theplurality of sets of hardware resources, and placing a second set ofcontainers, of the plurality of containers, on a second set of hardwareresources of the plurality of sets of hardware resources.
 19. The methodof claim 18, wherein the plurality of sets of hardware resources eachinclude a respective node of a virtualized environment.
 20. The methodof claim 18, wherein a higher score is associated with a greater measureof resource utilization of the plurality of sets of hardware resources.